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capabilities Tin the SCHOLARyQAI. system centering in three 
main areas: |{J^ implementation of two presentation 
• strategies i|i SCHOLAR (Tutorial mode and Block-Test mode) 
and a compa'rative evaluation of these two modes using 

high-school students as subjects; (2) "initial study. based 
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' on an^rtysis of ^utorial diailoguei^ of how .to teach 



procedural knowledge interactively within SCHOLAR ,r and 
(3) addition of a module for teaching geography using the 
map display and related question-answering facilities 
recently added to SCHOLAR. 
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In this paper we^ attempt to analyze title strategies by 
which tutors adapt their teaching to, individual students, 
so that we can synthesize these strategies in 1 
\SciraLAR^ CAI ^system. To find out \^at ptrategie 
tutors use, we tape-recorded dialogues , between/various 
tutbrs and students on the topic of Soiith A^Herican . - 
geogr^aphy. Because SCHOLAR is a well?-de£infed pro^ra0, 



•it is possible to -analyze sucl:| ill-.defi^d naturaii^ 
data in precise terms, with respeqt .po fhe^^structu^e. \ 
and proce5sing of information in SCHOLAR. We analyzed 
the dialogues concentratin j on one aspect- at a time. 
Based on our analyses,^ we ]?ropose in this pape'r several 
hypotheses about how the .tmtor relates his teaci^ing to 
the individual student. W6^how how in modified form 
We have implemented some of^fcsse strategies in SCHOLAR. ; 
We^ further argue ,that the analytical method emplo^d. here 
could be extended to a wide range of "conversatibnair 
situations. This method (Dialogue Analysis) would permit 
psychologists t6 study questions * about *tha int 
aspects^ of liuman processing that cannot e^en be considered 
with traditional laboratory methods. 



eoljins^ A. M. Comparison of two teaching strategies in computer- 
^ assisted instruction. BBN ^Report No: 2885/ 1974. Submitted 
to Ino^tructional Scien ce. ^ 




Three experiments were run using the; SCHOLAR CAI rsystem V 
to teach geography U> high-school students, 'the • experi- 
ments compared a metthod of teaching derived from analysis 
of human tutors (Tutorial mode) vs, "a- method derived 
from programmed instructicav (Block-Test mode^^^^^^^^^^ the 
three experiments, Block-Test mode was systematically ^ 
converged toward Tutorial mode in order to pinpoint what 
aspects o£ .teachiilg strategy affected students' learning. 
Tutorial mode was significantly more effective in the 

, first two experiments, and nonsignif icantly in the third. <^J^ 
Th/e Results indficated that the major factor affecting 
st^udd^ts* leariing was the strategy that tutors use of 
revieWin^g the jlaaterial in greater depth on a second pass. 
AlloWiWg the students to' ask questions, and the tutorial 

,.st^;&tec[y for relating new material to the students' 
previbuy knowledge contributed only a smalJL amount to 



the 



©rignetti, I M. 
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on-line as 



if^erencei^ found in the firi^t two experiment?. 



775-781. 



C. ,\ Hausmdnn, C, and • Gould, .L. An "intelligl^nt"^ 
sist^nt and' tutor— NLS-SCHOLAR. In Proceed ings Vof 



the Natit>n . il Com put er Co nference, ^San Diego, California, 1975,, 



(Partial support) 



NLS-SCHOLAR is a prototype system," that uses Artificial 
Intelligence ^techniques to teach computer-naive people 
how to use a powerful and complex editbr. It represents 



ERLC 



a new kind of Computer Assisted Instruction (CAl) system 
that integrates \systematic teaching with actual practipe^ 
i.ie.f one wtyLch can keep the user under tutorial supet-^ 
vision while allowing him to try but what he learns on 
the system Jie is learning about. ^ 



NLS-SCHQLAR caii ^lsb be Used-aiS^^^ on-line help system 
outside the tutbri/al environment^ ih the course, of a 




user's actuarr 
assistance wi 
nbtion of CAr 



oa^k. This capability of combining on-line , 
training is an extension of the traditional 



The techniques used in NLS^SCHOLAR are general and 'can 
be applied to a wi-de variety/^ computer-related^ activities. 




' > . ' ■ 'm - ' 

Collins/ A, M.VfHJarnock, H. ^ Aiello, N., ^nd Miller^ J4. W. 

Reasoning from incomplete knowledge. In D. G. Bobrc^^^^nd 

A. M. Collins (Eds.)^ Represe ntation an d understanding;. , 
New York: Acadfemic^ Press ^ 1975. . ^ ' 



The paper describes how people use a variety Qf plausible^, 
but uncertain^ inferenceis to answer questions abput which 
their knowledge is incomplete. This kind of reasoning is 
described in terms of how it is being implemented in the 
SCHOLAR CAI system. The paper alsQ-^ iows ho w people can / 
pe taught to reason in this way^ u^ing a Socratic tutorial 
method implemented in a system like^ SCHOLAR. . 
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Collins^ A. Educl^tioh and understanding. In D. Klahr CEd.) 
Cognition and ihVtruction , Hillsdale^ N. J.: Erlbaum ' 
Associates^ 1975. \ - 

* * \ 
^ This chapter c<m^ on chapters by Just and Carpenter 

and by Simon and Hayes on teaching understanding, skills 

The chapter argufes that the most important aspect of" 

und^standing is Uiow people use their knowledge about 

the world to fill \ in ""the information th^e tea/t assumes. 

Thus there can be no easy way to educate people to 

understand^ beca,usd they need to be taught both a huge 

^amount of world knowledge and the understanding skills 

to use that knowledge effectively in reading. 

Collins,, A. and Grignetti^ Mi C/ Intelligent CAI . 

BBN Report No. 3181, 1975.'^ To be submitted to Sciences ^ • 



This paper describes thW cajpabili ties now available for 
building intelligent, ti\toria^ CAl 'systems as exemplified 
by several/ systems including Tutor-SCHOLAR, MaprSCHOLAR, 
NLS -SCHOLAR and SOPHIE, ^he systems illustrate^how a' 
: variety of sophisticated t^chniq'ue^ can bfe used for 
tutorifig different kinds ofi knowledge by carrying on 
dialogues in natural language. The systems have been 
developed to explore how to provide each student with 
^^his own pei^sonal, expert tutdf , 



if , 



Qollins, A. , Pew, R. Iff. , arid Adams, M. The effectiveness 

of an interactive itiap display in tutoring geography. In 

j)reparatiorf. ^ * 

\ -".^ ' . ' 

This p^per will describe the Map-SCHOLAR system and 

experiment that compares how well s^tudents learn in 

Tutorial Mode, using (a) the interactive map display 

of Map-SCHOLAR (b) a static labeled map, and (c) an 

unl abeled, map. The paper will also show how a ifew , 

method called backtrace analysis -can be used to 

pinpoint* thte effectiveness of different aspects of 

the tutoring strategy and ^the map system used in the"^ 

experiment. [ 
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TNTRODUCTION . ' ^ ^ 



\ 



* If , computerized inatrlhction is ever to have a large impact »on 

educa,tion| computer-assisted instruction (CkJ) sVst-ems ♦must have , the 

flexibility knd skill af a humafi teacher. In developing the| SCHOLAR 

« "'* **• ■ ' * 

GAI system Carbonell (.1) tpok'a first ^^tep toward .an intelligeat*;^ 

. ; ' ' ■. ■ - ' . . 

tutorial C'AI system. In SCHOLAR, ^knowledge •was not stored as" text,/ 
but- in an interrelated network of facts and concepts*, so that Ihe 



'Slat 
u^ec 



knowledge could be u;^ed i-n a variety of ways. In short 'the Attempt 
was to structure inf ormation like a huma'n knowledge so that th^ 
pnogj?am could use its knowledge as fle,xibly as "a hiiman tutor does/ 



In this pap'er we will disciuss/ the structure of the SCHOLAR 
system, some of tR%~l7aTS that ^^he^p^ for intelligent CAT has • 

been realized in current systems', and finally what is ppssibl-e 1' in 
the na^r future 'toward building intelligent tutorial systems^ ; * 

The Context of SCHOLAR in CAI J 

^ ' ■ - . ' ■ . - ' * 

Prior to Carbonell/rf S.CHOLAR progr^ai, C^I had proceeded albng 
several lines. Bryan (2) distingui^heV^ree broad, categories • In 
the first, ^d-hoc CAI, the *§tudent is given fuli. control. Of the\ 
computer with a simple programming langu^\ and perhaps a. series of^ 
tasks to perform, ;%0G0 (3). pr^ovides one of . the most - interesting' 
educational environments of this kind and indeed children learn some 
important cognitive" skills in " working', with ^ LOGO, The. -second 
category , is games^ and siWiulatipn, where ^ the student .learns 
indir'ecjtly while , participating in- tbe game or simulation, ^ the Plato 
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-. . ■ • . ' ' ■ ■ ' ' * • , 

system's "How the west was won?"* (4), I's an excellent example of such 
*i •■ .' ' ■ • ■■ 

a system where children leacn the arithmetij>'operations in playing a 

variant of "c)?utes and Lad4trs-" Bpth these foijms of CAI are highly 



interactive, but they are limited as tea'ch-ing/ 'ftethods to' certain, 
vkinds of "knowledge. ^ ' -11 

The third category Bryan called . controlled^ learning- Host 
programs in this category specify the po&sibie sequences through a 
program, where , different branches are taken , deperrdlng .on the 
student's responses to questions or problems^ The sequence -a 

■ ' \ • • J \ ' ' ■ ' ♦ ■ - ■ ■ 

student follows/is usually deterministic, with a branch for each 
anticipated class of responses by the student (sometimes baaed, on a 

■ » ■ ■ ^ * ■ * ' ' 

. keyWprd he mighfe give). Some ijigenious programs can.be written in, 

this wayv su^h asi the Socratic system (5") or the chemistry programs 

in the Pl^to System (6), but there are som^ inherent -limitations to 

this / approach. - The student can^ not use hatiiral language in his 

, , ^ ■^ ^ ^ ♦ V ■ . ■ . . . 

responses, • and: canncttV ask any but sprecif ically anticipated 

^ ^'^ ' ^- . ■ , ^ \ ' . ' . . ■ : . ■ ' . ' 

questibns. The teacher ' has a - considerable burden in tjae preparation • 
of questions, answers , '^Kfeywords , and branchings. From a system 's 

' • ■ . ". ■ ^ • ' ' . . , ... ' 

point of view, the system has no real initiative or decision power 
of it^s own, ^nor any knowledge that is" available other than at fV|.xe 
.j&ointSif in the sequence. 



/♦ The rigidity ^bf ^this--latter approach led to the development of 
y , ' • • « r- , ' • ■• . ■ 

"generative CAI" <7). The firsfe CAI programs were based on 

mathematics^ and other weLl-defiried subjects; wh«?e problems could be 

generated and ^answers evaluated by "algorithmiq manipulation". This 
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was a ma'jor step toward representing a program 'sMcnowledge so that 
it cou^d be . used in ,a variety ^ of ways. But the knowledge vaS 
limited to algorithmic domains. 

Carbon^ll s contribution j^ras to apply the idea of generative 
CAI to ill-defined verbal knowledge, in particular the domain of 
^e6gr.ap4i^. do this he used/A a ^^^mantic nqtwork • (8) for 

representing knowledge about the, world. In a semantic network, 
information ' is stored about concepts under different entries, much 
as in ^n enbycldffedia , ^but .'according to a w^ll-defin^d format. 
/Every concept used to describe a given concept can . itself . be 
descrih^ elsewhere in the network. Thus the program can know about 



every donjcept •it , uses. 

Wliftn information . is represent^Jd in the general format a 
semantic, network, -then the stfrirffEtire or tlfe network can be used by 
different procedures to foriniijtate questions to the student, evaluate 
his answers^ 'answer his que^^^ohs*, makeVijvferences aed computations, 
select new topics for discussion , -etc. Filing the information from 
the specific . use^ it. is fo be put makes it available for Use in a 
Variety of ways. In this paper we will t^ry to show some ofV the. 
power. Tor tutorial interaction? ^ that can be obtained by such an 
api>ri.oach to GAl. ; \, 



The Original SCHOLAR Pro^^ratn 

Figurel presents a short) protocol with SCHOLAR , shoWing some 



t^ protocol with S 



of the original system ^s capabilities. SCHOLAR selected topics on a. 



9^. 



J 



WHAT. ARE-THE PRINCIPAL CITIES OF BRAZIL? 
•RIO AND MAYBE BUEN(is AIRES OR SA^ PAULO* 

RIO DE .JANEIRO AND SAO PAULO ARE CORRECT. YOU MISS*«LLED 
SAO PAULO. BUfeNOS. AIRES IS IN ARGENTINA, NOT IN BRAZIL 
•THE OTHER. PRINCIPAL CITI'ES OF BRAZIL ARE BRASILIA/ PORTO 
ALEGRE, SALVADOR, RECIFE, SANTOS, MANA06, ANDBELEM- 

TRUE 0^ FALSE? THE LANGUAGE IN BRAZIL -IS. FRENCH 

•Q/A». . ' ' ■ 

^ „ ' ^ *■ „ . 

DO you WANT TO CHANGE THE GENERAL MODE OF THE INTEMACTION TO 
QUESTION-.ANSWERING MODE? PLEASE ANSWER Y QR N. 

ok/ MR. JONES. LET'S GO AHEAD. 
•WHERE IS'^RASILIA?* ' . • . 



BRASltlA.IS IN CENTRAL/ BRAZIL. IT I 
HIGHLANDS. . 

•TELL ME MORE ABOUT THE LOCATION OP BRASliLIA?^ 





IN THE BRAZILIAN 



THE LOl^(SIT.UDE IS 1^8. 



THE LATITUDE OF BRASILIA IS -16, 
•WHAT IS LATITUDE?* - ^ 

^A^MTITUDE IS 'an/angular DISTANCE- FROM THK EQUATOR. IT CAN 
BEllgNSIDERED A (PART OF A LOCATION. • '^^ ^ 



•WHAT IS THE MOs\ IMPORTANT DIFFERENCE BETWEEN BRASILIA AND 
RIO*' ^ ■ * » , ' ■ |- 

lilS IW^^^ DIPFEREN'CE IS: BRASILIA IS IN /CeNTRAL BRAZIL. 
.BUT RIO DE JANEIRO IS IN SOUTHEASTERN BRAZILT ' 

•Is THE CLIMATE OF SAO PAULO. TROPICAL^ ' ' 

NO, THE CLIMATE OF SAO PAULO IS TEMPfiRATE. ' . 



Pig. 1 Dialogue with the original SCHOIAR 
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, random basis, ' wel^hted\for Jimportan When it printed an asterisk, . 
tlie .student ■« could ''re^dn^ to the question by SCHOLAR, direct a 
question^t^ SCHOL/yR , or y^sue comraand such as halting or "chanpiriK 



^th^^.mode pf interaction J/ The student returned control to SCHOLAR by 
1>yping 'arifc-dster^iskN^ - V ^ 

.icmLkB'^s /^v^!^8t^^ wer^e .constructed ff'Sfc the information 4m 
the sem^nt^c het^Koj;^ were not prestored. The student's answers 



V 



were/. eva^J-uated yby 7diffe prg-c^edures with respect t& ' the 

network, d^ci-dihg^ whether the particular a^rfswer 
wa3 •ooi;^|^;et,'W^ parl^ly. -cSlRreGrTt-^-^ approximately^ correct. 

■.Student j's qi^^'lt™ analysed syntactically tdVletermine What 

/.Information^ and the semantic network searche<? to find 

the infAi^m^ Different ^computations and inferences were 

/appljLed,. If'/ ^ hot stored directly. ^ All these 

operat^aoria: were | car-ried ; out by^ procedures /which operated 



inde|peh|fently of tie specific information"^ that was Involved. 

I I^^^^^ 1 the\que3t;ions by SCHOLAR illustrate di/ferent klnds\ 

of qu^tions that dould be generated. ^ After the sd^cond question- by 
SCH(|L^ 
mixddl-i 



the studei\t chaoged the mode of * interaction from 
iitiative rao^.e, where SCHOLAR aske^ KiJ questions, to 



que£ ti|QH-answering mod'^> Whesre^^CHQLAR waited for questions from' the 



sti^cei?t;V Other modes\ described • below- have since be«n added to 
S€HC|LA^.\ Secause the student can control the way he interacts with 
the syst^ete, he. can^ choose the mode of interaction tha;t he finds most 
effecti^l^.\ This is^ one of the. important , w^ys such a system' 
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personalizes instructions ^ — . 

Tn^ student then asked a serines of questions to •clarify a«d 
extend the information given to him iibout the cities' of Br^ail. 
When /the student wants more information* about something such/ as 

f\ ■ • ■ 

ilia, /he 



Bra 



exyplaine 
Individ 




/ 



can ask specifically what he wants to k^rfdw. When Ijje 



doefen't urid^rstaiM a wprd, such latitude, he ' ban have it 

In. this way the knowltf'dge taugh^tcarj be 'geared the 
studeht ^back^ound, so as not «:'to /repe4l^ w^t\ he already 



knows ybrf go ov^i^ his head. This i^ import^h't to maintaining a 
'studeni '9 motivatio\i to learn. 



!.■ 

hef two question^ about tJje locatl^ti cff Brasilia illustlrate how 

a tu|torial system ca^/^^oid, overloading ;j;he student with too mych 

- * * - ■ ■ ■'■ ■■' . ■ . / . ' ' ■ 

information at one time. Each piece .of Information in/ the netwprk 

• ' ' - - - ■ K 

tagged to ihdipate its relative- impbrtance* the program gives 

only the most ^importajpt. information at g^ny time, but t^ie student eah 

alwaya a9k for more information if he vslnts itv 

♦ . The^ questions about * the most#^ difference between 

Brasilia and Rio and^' about the clfiiiate of Sao Paulo illustrate the 
ability to i^^e a semantic network ,fo make ^ appropriate cbmputfeti^ons 
and inferences, * In the first' case there is* a procedure, for 
comparing two things to* find theM^ similardties and/or ciif f^^re^KCfes . 

, ; ^ ■ . . / : - . ' 

Each pl^operty of the two things is compared - in the orct^r of 
importances^ Here the mos^t important property on vifhich tbe^ two 
thij^gs differ is found, and ;^iven as an answer to the student. > The 
second case illustrates, a combination of two inferences, a deduction 
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and a ^^contradiction , 
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Nothing about climate 'is'^stored with Sao 

• ■ / " . 

JPaulo, but Sao Paulo is in the Br^?ili'an Highlands which has -^a 

te^peratfe clirtate. By comparing, tropical- and temper'gte, SCHOLARS 

'finds There a contradiction and CjOnciudes th^ answer is ;'no"A 

There are a large number of such Inferential strategies that, humans ; 

- / 

use, and anly somf of the more common ones have been implemented in 

. 't ' 

• - ' . ■ u • • • 

SQHOLAR. But information in SCHOLAR is ' structured .in sUch a way ' 

^ ^ ■ «. ' > ■ • . J 

that it is ipo^sible tb specify content-independent procedures fc9 
carry out different inferences. ' ' , 

:-This ^summarizes the i|aJor contributions of the orig^natl SCHoMr 



system- TKef^werfe^aJ^o^ to ttif o?jl^nal 

' ■ ' . " ■ • . . .... y ' ' J' . 'i'^-^^'^''^ . J 

SCHOLAR. First , the idfori^a^ion In 'the- program^ w re3j^|ted to 



static , veiV^al facts abput g'eog^ which are not very interesting 

in themselvvesV Second , the^pAograin was quitife restrict^di^ in its 



ability to „ understand 



i 



si^udent. answers and questions because' of its 
limited language processing capability. .Third, and | pefhaps most , 
important there was no teaching strategy; the .program merely 



generated questions rai^^lTriy or answered student questions, • We '.will^ 

' . ' ' ^ ■ .■ ^ ■ v" ■ ^ ^ : • ' ■ . ■ ■ 

try, to sh^W how later r^ystems haVe overcome some of the limitatjons 

Of ttie ^i^iginal SCHOWtW and> at> tlie same time exp^ited i^urth^l^ its 



potep^eial for tutori^liinti^raction: 
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. r TEACHING STRATEGY , 

Analysis of Human Tutoring » 

- . 

• ■* 

In the original SCHOLAfi thfere wate. no teaching strategy, but 

V • , . ■ - ■ . , y. 

SCHOLA|t SL structure njade it possible to model the way human tutors 
interact, with students- By collecting tape recordings of different 
tutors teaching the same kind- of information as SCHOLAR, it^was 
poss^-ble to analyze how tutors -adapt their- teaching to * the 
ihdiv^ual student (9): There were^6ur crucial aspect^^f their 
tutoring strategy, that were subsequently niTSd^led in SCHOLAR^ /They 
were: (a) the way tutors se/.e'ct tot>ics, (b)- the way they . interweave 
questions. and presentation; (c) their reviewing, and (d) their erroV, 
correction strategy. - ^ ' - 

■ ' .. \ - ■ ' ' ■ ' ' ' 

The t^t)pic selection "s^ategy used by, tutors produces a 

structure of topics and subtopics like atv^outli/ie for a cotirse. For 

example, ,th,e tutor might start off with/a quest?lh like "Do you know 

any geographical features of South America?" -If the student gives 

Cape Horn, ^ for e^ampXe, then the tutor would discOfss Cape Horn^for a 

while, including perhaps th>e' Straits •t>f Magellan "as a subtopio. 

Aftw covering the most important information ^out> Cape <I^drn,/^ the 

tutor would then ask about other geographical features^ like the 

a . . ... 

Amazon or the Andes. Each of the^e would be discussed briefly until 
the lii^jor geographical, features are cov,ered^ at which point tiie 
tutor wouia pick a nev{ topic such as regions, or ' countries? Thus, 
the topics and subtopics form a nes,ted outline structure , with the 
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tutor probing .a litfele^ waty into each subtopic, and then popping up 
to the previous topic when the important info'rrnation is exhausted. 
The/better the tutor,' the more struct)ure there i? to the discission. 



.'The way thp tutor interweaves questioning and presentation is 
ssence of how the^ tujor^ rela^tes his, teaching to the individual 
student. ..i;he dialogues showed .th4t the tutors' questions ^ occurt^ at 
the top-level and beginning topics in th^ outline. This it's because 
j^^-fe+i^e- tutor- '3ta^^^ asking questions to find out ^hat the student 

^ • . . . r. ' ' ^ . ■ 

already knows, and then presents^ rtew material that i% related to th^ 
student 's /previous knowledge. Th'^ object seems to ^e to tie as much 

■ ■ •. • ■ ^ ■: - ■ ' . / ■ 

as the student canoasi^imil^te into the structure of his 



inf ormati 



'In 



pre viou^ -knowledge (10) 



Another impdrtant aspect of t 



Ject of th 



le tutorial strategy is reviewing. 



Int t-he ^dialogues the better? tutors went over the material on ;a 
SecoKid pass, askfing abou.t things tjhe student didn't know the first 
^in^e through^,* ai|id adding more det^iil to the structure of infortnation 

■ The tutorial method as' a wholei 



buiit up on th^ first pass-l 
reflected" a strategy. Norman (It)) 



refers to as "web teaching"^ ^ 



re 



the Jteacher firs't tries to est^bliish ^a framework of basic knowledge 
and tijhert fills In mo,re and ioie detail on subsequent passes , much 
\ like" a- spider spinning a web*. 

foji^.tl? aspect of . the /dilalogues important to individualizing 
Instruction the way tutors correct student 'errors . When students 
make fe confusion between two copcept$j the better tutors try to 
j provide distinguishing praggr^tiTe? i between, the concepts for the 
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student .. ' For example , , when one student confused Ecuador and 
Colombia, the tutor pointed out that Ecuador is a much smaller 
country and th^rt Colombia is connected to Panama:;, By providing 
distinguishing characteristics, the tutor is giving the individual, 
the 'most relevant information for remembering the distinction in the 
future. ' ' ^ ■ 



Tutorial Mods in SCHOLAR . \ ♦ * - ♦ . ' 

• .-• ' • ' ■ ■ ■ . ■ " / . '. ' 

T'hese four aspects of . the human tutoring^ strateigy were 

developed in a mode called Tutor-SCHOLAR ('9). Like the human, tutor, 
Ju.tor-SCHOLAR selects topics in order of importance, ^ arid, goes into" 
dept-h . on those topics the student knows something about. It starts 
o.ut asking questions alv;ays probing^ deeper untiJt the studeftt doesn't 
know an answer or the time is used up for that topic. When the 
studefit. misses a question," it presents' .a little .related information 
for the student to assiThilate before going onto the next topic. It 
. allocates its time between a first pass and a re view pass.*- On the 
review pass it skips oyer what the student ^new earlier: However,, 
it asks about everything i;h6 student missed, or that was presented 
eai^lier. Anything the studept remembers from the earlier pass leads 
: ^^''^ ^^^*^H®si''"/i"to dep Oft the revi#w. pass. .< 

J :. When ; the student suggests an answer that differs from what is 
.steered, Tutor-SGHOLAR tries bo find a basi% • for rejecting the 
is^lldent 's answer. This involves an inferential comparison between 
- wt?;at thp student says and T/hat is, stored. If it can' find a- 
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contradiction, the program points out the basis Tor rejec^inij the 
student 's ■ ^answer * plup anything it knows 'about the relat^ipn' Of t^he 



g^udent^s anawer to the object in question. In this way it tries to 
gi\ve the student, a^ basis .Sfor remembering the distinction in the 
.fuiiare,' ■■ , • ' ' ' ! W 

y Figure 2 shows a transcript betweert tutor-SCHOLAR anc^ a >|^ive 

stade^nt who was testing out the .system, It^shows t^e kinds of 

' * 1' • « ' ■ ^' •■ • ■ 

. - . ■ » '•1; r:/; ■ • » 

mistakes SCHOLAR makes in language^;;^ftderstHndtTTgT — Eut it' also shows 
the interactive flavor of SCHOLAR with a tutoring strategjL-^ir^ttng 
the conversatioq, * ^ . ; \ « 

, SvstematlG Evaluation of Teaching Methods "* . , 

BecAuSie the inf oriaatioil in SCHOLA|^^is 'distinct from the 
procedures . used for* teaching thafTiiTermation, it i^g^'^i^ssibiL^j to 

vary the^^teaching strategy -Vhile holding the ' infprrriatTbn " constsjftt. 

■ • • • '•■ * ' ' ■■ ^ \ , ■ - ' 
TThis makes ' it possible to compare different -^'li^achilng methods in a 

■ . • ■ ' . ', ' • ' ■ * ■ ' ■ ' • ■ ' ■ 
systematic manner. To evaluate the effectiveness of- Tutor-SCHQLAR 

' ■ ■ * ' . - . ' n ' • 

we compared it to a teachin^g strategy derived from prograrnmed 

instruction, which was implemented in a mode called Block-SCHOLAR. 

• • . . If ■. , , . ^ ^ 

Block-StHOLAR selects topics in the same order as Tutor-SCHOLAR , but 

unlike Tutop-SCHOLAR it first presents miterial and then asks 

' . • ' - i 

questions about the material presented, , 

' ^ , . * . • . • '. / ' 'a ■ ' ■ 

- : ' ^hree experime_nts were run to compare Xutor-SCHOLAR and 
oHloQk-SCHOLAR (11). The comparison was made by teaching students 
the same domain of information with the- two vers iorm_-Df SCHOLAR, an^ 



LET'S BEGIN BY TALKING ABOUT THE LOCATION Uf i BRAZIL. T^E 
OR FALSE? BRAZIL IS IN NORTHEASTERN '^S^UTh |m|.RICA. 

•THAT IS' FALSE* -^.ll f 

NO, BRAZIL IS ift NOf^THEASTERN SOUTH 4|MER:|:i|' 

Tpf EASTERN COAST OF BRAZIL IS THE ATLA^ilfecl.^ THE WESTERN 
BORDERING ^ COUNTRIES ARE PER^ AND COUdifBi^; THE SOUTHERN 
BORDERING COUNTRIES AftE URUGUAY AND ARGE4t|IS|. 

^ • HI < A 

Because tfi.e student ntissed the jqueM^n^ SCROLAR presents 
some related information about the ^^Sex^ion ^of Brazil^ , 
and th^n goes on to the next topiall^ . i 

"V ■ ■ ' ' ' ' ■ ' 

'NEXT WE'RE GOING TO DISCUSS iTHE TOPOGRAliftl 
\PRINCIPAL RIVERS OF BRAZIL ARE : 



•THE AMAZON AND THE ORINOCO.* 



BRAZIL , 



THE 



THE^AMAZON IS CORRECT. THE ORINOCO ZS' ilN^J^' VENEZUELA AND. 
COLOMBIA., NOT IN BRAZIL. THE OTHER P|l*iqiPAL RIVERS OF 
BRAZIL ARE THE TOCANTINS, THE SAO FRANCISCt). THE RID NEGRO. - 
AND TiiE MADEIRA. • • ^ ' >: ' 



THE SAO FRANCISCO IS IN EASTERN BRAZIL.! iie IS XN THE* 
BRAZILIAN HIGHLANDS. THE RIO NEGRO IS IN TH^^ AMAZON JUNGLE. 
IT IS IN NORTHWESTERN BRAZIL. 



Because the student knew about the Amqz<)^\ StUOLAR takes 
it up as a topic for a while, ^ ' 



CORRECT OR INtORREgfT? THE AMAZON IS NAVIGABLE^;:- 



•POSSIBLY^ YES 
m. THE^ AMAZON IS NAVIGABLE 



SCHOLAR misunderstood the ''possiblt^'rand-t^veated ihe 
answer as wrong. ' * 

THE AMAZON IS IN THE AMAZON JUNGLE. IT IS IN aRAZIL: AND 

PERU. THE PRINCIPAL TRIBUTARIES ARE THE .R'la NEGRO' THE 

MADEIRA,^HE TOCANTlNS, THE TAPAJOZ, THE PCIRUS. .AND THE 
JURUA. - . * 

WHAT IS THfe LENGTH OF THE AMAZON? ' -fi 



Fig . 2 dialogue with Tutor-iSCITOLAk 
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23 



V 



•IS THE AMAZON A LAKE?* \ - 
•I DON'T KNOW IF THE AMAZ^H^N IS A LAKE, BUT IT ts A RIVER. 

SCHOLAR found that both lakes and rivers ^ere Water^ 
cso it couldn't reject the idea* 

'NOW' YOU CAN EITHEl*? ANSWER MY QUESTlbN OR ASK lA NEw' ONE. 



•WHAT IS THE QUESTION?* " /■ 
WHAT IS THE LENGTH OF THE j^f^^iON? , 



•5000 MILES* ^ /J"- ■ ■ . -"^^^^^^ 

NO. THE'LENdTH OF THE AMAZ.ON IS 4^000 MILES, 

' - ■ . " .' ' , 

Having exhauste^d its allotted iirhe to rivers^ SCHOLAR 
went' oHto tJ}e next sy/btopic under topography of^Brazil. 



TflE PRINCIPAL MOUNTAINS OF BRAZIL ARE --— . 

*I AM NOT SURE, ARE THE ANDES' IN BRAZIL?* ■ ' 

■ 

IN WHAT YOU SAY, I DON'T RECOGNIZE THE WORDS "I", "AM" AND 
"SURE". TRY AGAINj. 

'*ARE THE ANDES IN BRAZIL?* 

NO. THE ANDES .ARE NOT IN BRAZIL, \bUT IN VENEZUELA, 
COLOMBIA, ECUADOR, PERU, BOLIVIA, CHILE, \aND ARGENTINA. 

FEEL FREE TO ANSWER THE* QUESTION YOU DIDN\T ANSWER BEF^Oi?E, 

•BRAZIL HAS NO MOUNTAINS.* 

NO. I DON'T KNOW WHAT YOU MEAN BY NO. TH^? CORRE'CT ANSWER 
IS THE BRAZILIAN •HIGHLANDS. 

SCHOLAR didn'-^underistdnd jthe answer, ^ut proceeded quite 
reasonably . - \ 



THE BRAZILIAN HIGHLANDS ARE IN SOUTHEASTERN \ BRAZIL. THEY 
ARE NEAR THE ATLANTIC. 

WHAT IS THE PRINCIP'AL PLATEAU OF BRAZIL? -. 

The dialogue continued diis-cus sing differ erit aspects of 
Brazil, but we will break here to the beginning of the 
second pass. * " 



Fig.' 2 Diaioqu'e t^ith Tutor-SCHOLAR (cont) 



NOW WE WILL GO OVER THE MATERIAL . AGAIN. LET'S BEGIN BY 
TALKING ABOUT THE LOCATl^ON OF •^BBAZIL. 



PLEASE INDICATE IF/THE FOLLOWING^ ^STATEMENT - IS CORRECT OR' 
NCORRECT; BRAZIL IS IN "NORTHEASITERN" SOUTH AMERICA. 



*TRU^E* . y 

THAT'S RIGHT. .^BRAZIL IS IN , NORTHEASTERN SOUTH AMERICA. ' 

^Because on thia p-ass the student was correct, SCHOLAR 
asked questioxis about the information that was presented 
the. fi^st pass. ^ ' t 

THE EASTERN COAST OF BRAZIL IS --".^ - V, • 

*ATLAN.iiC SEA.* * * • 

VERY GOOD. THE ATLANTIC IS CORRECT. 

THE WfiSTERN 'BORDERING COUNTRJjlES OF BRAZIL . ARE 



The 'dialogue continued through the secop^S'^pdss. 



/ 



V 



. \ 

\ 



Fig. 2 Dialogue with Tutor-SCHOLAR (cbnt): 



14 



2^- ' ^ 




- BBN RepoPt No. 3181 ' ^ Bolt B^rariek.and Newroan Irie, 

• '* , \ ■ * ' ^ ■ • 

measuring ttjeir learning by comparing" p're-^est and post-test scores 
. o« the material covered. In the . three experiments, Block-SCHOLARv 
'^as systematically converged toward Tutor-SCHOLAR in . o^rder to 
J^npoint what' aspects of teachtng strategy affected student 
i®|arning. Tutor-^SCHOLAR was significantly more effective in the 
|Fj.rst tijro experiments, and nornsignif icahftly in \the , thirds The,, 
j^esults indicated that the ma^or^ factor affecting student 's learning 

f was -the strategy that tutors use of reviewing the material i^n 
;-<greater dfepth Qh^ a, second pass. ' Allowing the student's tp ask 
o7 questions, and the tutorial strategy of relating new material to the*^ 

f ■ /• ' ■ ■■ ' ^ ^ . ■ - ■ ■ . ' .\ 

i student's previous knowleS-ge contributed a smaller amount to the' 
greater effectiveness-.of Tutor-SCHOLAR. - ^ . 

The fact that SCHOLAR can be used^ to test particular aspects of 
teaching methods/ ntakes \ it potent iailly . a valuable tool for" 
educational; research.- The possibility of trying outy single 
modifications in teaching strategy to see their effects, on stWa^it's 

' ' ' " " ' - ■ . . ■ . . ' ' ^ 

learning rate is unique. Human teachers of course can/ mdke such 

■ • . ^ ■ ■ . ' . ' " ■ «^ * ' ' ' 

modifiGatioris in their own teaching' strategies , but -there is no way 

V ^ . ■ 

to Gontro;L all the other factors that might vary< as - they ' change 

. t/ - « ■ ' .** * ■. . * 

strategy. However, ^any spetfifio version of SCHOLAR is a fixed 

■ - - ■.■,*■<;.■■ ^ 

system, and so> an unbiased comparison can be made using any number 

of subjects. In this way the accumulation of systematic .k^iowledge 

about; teaching methods , can occur. ... . > ' 
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'UTORING DIFFE^IENT TYPES OF KNOWLEDGE 



• On^ qJr the limitations of t'lie original SCHOLAR wais "that it was 
rdst'riotecl to teacljiing yerbal facts about geography. The ;SCHOLAR 
system itself has been, extended tV ■ e'ncompass two other kinds of 

knowledge^ ""visual knowledge gbout maps in the geo^aphy, domain-, and 

■ ■ " . ' ■ ■ ■ ' . , . ■ . r ■ ■ * ■, ■ . ■■ •• 

procedural knowledge' about how to use'a computer text-editing__system- 



called NLS (12). A related program Galled SOPHIE (rS) tutors the 
diagnostic skills;, needed in -' "electronic troubleshooting. ^In^ this^ 
section we will try to shoij the generality of this approach to CAI^ 
and also some of the specif iq adaptions th"at have occurr^ed in 

applying it to different ^domains- of knowledge. 

. . • - ■ ' ' ■ "• . ■■ V ' .. 

Tutoring Visua^l Knowledge .. ■ " ' ' . J ' - 

■•>■■■■;■ ■ * ■■ ■ '. { ' ■ 

In order to explore the tutoring of visual information iri an 
integrated 'manner with - verbal information, we ^developed a 
Map-SCHOLAR System (14). .The^ystem can discuss with the student 
different maps "that change idynanjically according to the costext of 
the discus^sion. To do this a graphic structure was created- whic 

parall^fls tt?e struGtu^e , in 'tjhe semantic network. The elements in 

' , •,, ' • ■ ■' ■ ■■ .■ ■■ • ' . ■ 

J:he map display therefore can be referred to either by their na,me, 

ort>" by pointing to^thera, or both. Map-SCHOLAR both asks aft'd answers 

map-related qiiestidns and proyi'des relevant, map :j.nf ormat ion when the 

Student-^ makes a misfaife; It has al;^ the ■capabilities of 

Tutor-SCHOLAR with the .addition 'of the. map capabilities . ] • * 



r ■ / r 
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- » ■. , ' • • 

• *' * * * , 

■ ■ ■ ' ' ■ 

FigVH»€ 3 illustrates some of the variety o-f" interactiohs 

•pdssible witK Map-SCHOLAR. The first pahel illustrates how 
■ • I" ■ . ■■ • 

. Map-SCHOjLAR asks map-related questions, e-.aluates the student's 

answers, and corrects, his mistakes. First, the dots indicating the 
•> .location of the cities appear unlabelled on the map of Brazil and 
. -^gtart blinking. Then SC HOLAR ask s i-H p .- tndnni- > ,> .;,nu».^ 1 1,^ hiinklng 
citi-es, . When he responds," it teils him which ajiswers , were correct 
and prints their names; on the map. Because he -erroheously gave Lima 
as an answer, Map-SGHOLAR pointed oiit that L^ima is .in Peru and 
• showed where , Lima 'is on the map.' Thus Map-Scholar gives both veri)al 
and visual feedbacktolhelp.the student learn the material. 

^ Th© secpnd panel shows how context affects the map display. 
The -topic is the Amazion, which appears, in this context on t'he map of 
•Brazil, the overall topic. The question asks' the studfeht to point 
at the cities along the Amazon in the order named. The dots for tl)e . 
, three cities first appear unlabelled , and -only after th^ . student has 
answered do the labels appear, Unlike 'the first' map, • this map shows 
only the cities on the Amazon and their relation to the .Amazon. • 
There is also expansion of detail so that a city like B*elem, which ^ 
was -not .important enough to be included among'^ the cities of vBraziV". 

, is included with the , subtbpic of the Amazori. Beeause the map 

.. ■ ■ -i- ' - ' ^ ■ ' ■ . - ■■■ r . ■ 

chahges dynat^fcally as the context changes aijd as the .student 
■ ^ -< — .■ ■ " ■ ■ . ■ ■ ; ■ 

interacts^ with the system, the students attention is focused on the 

: relevant information exclusively and questions can be posed in a 
"visual form not possible with a static map display. 

ERIC . : : ■ 



NOW LET'S SEE WHAT YOU REMEMBBR ABOUT THE CITIES OP BRAZIL. 
PLEASE NAME THE PRINCIP;^L CITIES OP BRAZIL NOW -BLINKING 
ON THE DISPLAY' SCREEN. - , 

♦MANAOS SAO ^AULO AND LIMA* , ' 

f 

^ MANAOS AND SAO PAULO ARE CORRECT. LIMA 19 IN PERU, NOT IN ^RAZIIr. 

THE PRINCIPAL CITIES OP BRAZIL ARE BLINklNG ON THE SCREEN. 
' • • 

RIO DE JANEIRO IS IN SOUTHEASTERN BRAZIL. * 

IT IS ON GUANABARA BAY AND Tlfe ATLANTIC. 



fBR^SILJA IS IN CENTRI\L BRAZIL. 
IT IS IFAP^ FROM THE OCEAN. 




RECIF 



0 0£ MfitlRO 



Fig. 3 • Interactions with Map-SCHOLAR 
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\PIiEASE POINT TO THE FOLLOWING CITIES OP THE AMAZON ' 
tN THE ORDER IN VWICH THEY AP|»EAR BELOW; 
ANAOS^ BELEM^ AND IQUITOS 
f*T FORGET TO LET ME KNOW WHEN YOU ARE FINISHED^ 
SASE REMEMBER TO TYPE ATI ASTERISK. * P0LI|0T7ED BY A 
^ lAGE RETURN. * . , ^ - V 



MAHAOS IS COR^CT'. YOU DID NOT POINT AT BELEM AND JdUITOS CORRECTLY. 
THE\ CITIES OF THE AMAZON ARE ^ BLINKING OS THE SCREEN. 




Pig. 3 Interaction's with Map- SCJfOLAR (cont) 
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0 



BY TALKING ABOUT THE TOPOGRAPHY OP VENEZUEI^A. 



THE PRINCIPM, LAKE OF VENEZUILA IS BLINKING^ PtEASE' NAT^ IT. * 

V *I DON'T KNOW* • 

THE PRINCIPAL LAKE OV VENEZUELA IS -BLINKING ON THE SCREEN. 

\ LAKE MARACAIBO IS IN NORTIW^STERN VENEZUELA. 

THE PRINCIPAL MINERAL IS OIL. ' 

♦IS THIS COUNTRY COLOMBIA?* ' 



*WHBiR^ ks THE CAPITAL OF VENEZUELA?* 
CARACAS IS BLINKING. ' 



1 




\ 



Pig. 3 livteractions wi^h Man-SCHOLAR (cont) 
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The third panel shows the system's potential ^for tutorial 

interabtion. When the\ stu^e(nt didn 't know about Lake Maraqaibo, 
' *■ ■ . ' . • , ■ • ' ■ * . • 

Map^SCHGLAfl ^howed it on the screen and added the related vj^rbal 

information about the oil there. The two question^\ by the student 

illustrate some of the j^ariety of questions the student can ask 

Map-SCHOLAR to 'jjjlarify or amplify the information given -(14). The 



first question involves both pointing kt antJ naming Colombia to aslT 
if that is the country near Lake Maracaibo. The second question 
shows another kind of map question. The subject verbally asks where 
the capital 6f Venezuela is, perhaps to find out how far away it is. 
SCHOLAR figures out semantically that tlvb capital Is Caracas, and 
then visually shows where Caracas is^ bx. blinking it (it is the 
double square). These examples illustrate some of Jfche power fop 
tutorial interactlan that can be obtained by ai close int^gratiort 
between semantic and visual knowledge* 



1 ■ . X ^. 

There is now substantial evidence in the psychpi-ogicai 

literature (15) that people remember imageable material better than 

non-imageabl§ material. Furthermore, if they use ihiage strategies 

for^^membering (^16) , • they remember any giyfen information bett^er 

than if they use other strategies. Hence the fact tha^ information 

is presented visually should make it more memorable. 

As Collins .& Qulllian and Norman (10) argue, the best way to^ 



learn sotoething is to relate it ,as much as possible to whatever 

information one already knows. Hence, even non-^visual fliformafeion 

* -.•''"* ■ - • ■ • ■ 

like the fact that the Manaos has a tropical climate, will be 
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learnecKbetter,, if one can see where* Manaos is. This is- true for 

tv{o reasons. First, because when Mariaos is located visually, it 

will be remembered better, and so facts that are- related to" it will 

. also Se rememberexl better.- Second, if a student sees that Wanads is 

^on the Amazon^ then its climate can be related to any knowledge 

-aboyt the climate of the Amazon. Thus, information that se'ems not 

— depenj^ on — 4ffiag#*iy — should V) e — r e m o mb o r o d — better in — a — visual" 

I ■ ■ ■ . - . . - 

context. \ For -these reasons we expect the map facility, to have a 

/substantial impact on how much students learn. - ^ 

' r- - . ■' , _ . ■ . \ , ■ ■ . ■ 

We have conducted an ^xpjerim^ental evaluation of the map -system 
^using the co'mparktive method .described earlier (11). '^The test 
compared student 's learning with Tutor-SCHOLAR using the map system 
vs. labeled map vs. an\ unlabelled map. T^ experiment found an 

. • ■ ' ' ; 

advantage of the map system \ over either of the static maps. We are 

'^^d^. ■ . -■ : \ . . , '■ 

mMS a technique called "backtrace analysi-s", which involves 
^coiparing the^pecific- information each student learned with ' how 
%h^l inforKnation was discussed, in order to pinpoint what aspect^ of 

the map system led to better learning by. the students . 

' ■ 'T ■ ■ ■ 

Tutoring Procedural Knowledge ' 

NLS-SCHOLAR (17) is a prototype system to'teach computer-naive 
people how to use the powerful NLS text-editing system (12). This . 
teaching is accomplished, by presenting a Sequence of lessons. 
During each lesson the student may interact with the system by 
asking and answering questions, performing tasks which are posed by 
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.-.■<.* 

' the system, 'and performing/ tasks of his'own choosing. Tasks are" 
^ executJWl on an actual NLS system. .Those tasks which have been posed 
-are evaluated by the system, ;and the student is given encouragement, 
advice, li^d .assistance.' 

« NLS-SCHOLAR has been designed with the, belief that procedural' 

/ * - - > * 

knowledge ^ is best learned 'by doing'. It is, an example (18) of a 



new kind of CAI system th^t .intejc^rates systematic teaching with 
actual practice, ^i.e. ,/ one wb^ch can keep a student under 
"in^tellig-ent" tutorial suf/ervig^ion Ji»i(hile allowing him to try Out 
What he learns on thd-^^ery sjrstem yhe is learning about. Thus th$ 
system "knows '\^wfeat the ^^Vident is doing and can point oul: his 
mistakes, give iMpecifie ' help; show him how to tlo things and even do 

.i - :,^ ■ 

NLS-SGHOLAfi delivers a series of lessons designed for gradual 
Understanding 'of concepts arid commands. Within these lessojis. 

the system pauses to ask the. student questions and to propose 
editing tasks for> him to"" perfof»m iising NlyS. A student's responses 

to questions^and his performance of tasks are evaluated by the 

'• " • ■ , 

system and if he makes an error, the nature of his mistake is 

■ " ' ■ <* * , 

poi^ited out and apprppriate action iis taken. For example, • if a 
question is answered unsatisfactorily, NLS-SCHOLAR proFfoses another 
question of the sameskind. If a task is p^rforniedv- Incorrectly, 
depending on the magnitude of °the error, NUS-SCriOLAR either resets 
it for the student to try again, or asks him to pi*oceed and try to' 
fix his mistake, aided by the inforination NLS^SCHOLAR prbvidjes.. 
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. The wsejr can formulate requests in .relatively uncdnstrained 
:nglish. The requests can be questiohs abouty'^^ concepts or about 
he state of his work, requests- for help in doi^g task, or even 
fLS commands expressed in,^;nglish. The system is "aware" of what 
he user is currently doing so that his requests for help can /be 
nswergd within the c<^|itext of the problem he is working on. Thus 



LS-gCHQLAR not only tells him "The general prnn.e-.dune> 



-b«^ 



llso."In your case, what you should do. is 



, NLS-SCHOLAR has the ability to use the NLS file a student is 
' ' A* . ■ - 

jrrentlW: working^ on tp show him how to perform editing actions'. 

Ti|is gives ^the system much of the flavor of a human tutor, as if he - 

weire taking* the student s place at the terminal and saying "Watch me 

do it- forlypu". . 



The flavor of NLS-SCHOLAR is best conveyed by an anrtotated 

^9 ■ 1 r ■ . 

protocol, I jshown in Fig. 4, which, was obtained on-line using^a .. 

version of I the system. In the colirseof a lesson,, students learn 

•■ I I " ■ ■ ^ ' ■ " 

how to cHabge the .6ont^nts of a menu by performing editing. 

operations. n The protocol • starts at a point^ well along in the 

student's /iebrning of NLS — he ha^ bQeh told about NLS files, how 

to load theml \print them, delete ^nd insert statements^ etc. He is 

about to bd W taught how to use the Substrtute command to' effect a 

Change in the\menu. Notice that what the student actually typed is 

^ I w ■ " ■ . ' 

underlined andVxiiat our annotations appear in italics. .'^ 

% \ w ' ■ ■ ' 

■ \ W ^ ' ' ■ ' ■ ' ■ ^ • ' ' ■ . . 

Jhecapabiaktiea-of NLS-SCHOLAR allow students to learn from. 
explanation, iM^n by doing, and learn by. asking questions. Their ' 



BREAKFAST MENl 



JUICE 

1A ORANGE 

IB' GRAPEFRUIT 

CEREAL 

2A OATMEAL 

2A1 WITH RAISINS 
2B CREAM OF WHEAT 
2C CORN FLAKES ) 
EGGS • ' 
3A SCRAMBLED 
3B FRIED 

3B1 SUNNY-SIDE-UP 
3B2 OVER-EASY 



to 



3C BOILED ~ '• 

BEVERAGE 

1|A HOT "CHOCOLATE ' 
lia.TEA 

# lIBI WITH LEMON ^ 
' "- l|B2 WITH SUGAtlAND CREAM 
1|C' COFFEE -j 



SUBSTITUTE WORP IN STATIillEWT i 

Bcrore we end this lesson, I'd like to show yon how to change the 
content of statements which have already been inserted In your file. 

^You've learned how to correct err*ors by deleting an entire statement' 
and insertinc a now one in its pl^ce. The Substitute comnand may be 
used to change just a word or eveft a single charafctcr at a time* 'V6t 
example, here's how you can change a word in statement 2A1' so that Jit 
read3 WITH DATES instead of WITH RAISKIS. ' ^ ^ 



EDIT C: ISu )bstitute C: (W )ord in 
)(<CR>); 

<New V/ORD> T: {DATES)(<CR>) ; 
' <01d WORD> T: ( RAISINS) (<CR>); - 
Finished? y/Nj (Yc) OK: (<CR>); " 
Substitutions made: 1 ^ 
EDirC: (Do )ne OK: (<CR>); 



C: (S )tatement at A: (2A1 



HLS-SCHOLAU per forma a command for ihe student much ^ 
human tutor tsould^y ieanitig over the student and typing 
the oommand on the student^ s terminals Pdrentita^c^ enclove 
0hat^ the "tutor^'^ actuatltf Hupec'^, cepardtin^ it fram^the 
mchoee and-responaea printed by //Zr5. ' 

.<CR> is just a s-ymb^l to denote that the/'Hutor*^ depfested 
the return key m. i^he semicolon is ilLS^t echo to <CJi>, 



Pig. 4 Dialogue with WLS-SCH^ 



,1* 



You can see that I ve specified that I want to substitute one word in 
statement 2A1 for another; the new v/ord DATES is to be 
substituted for the old word RAISIMS. v ^ 

(There is no reStniction that the items substituted far 
one anpther be of the same le|igth.)* 

To do this 1 typed only 'Su M S 2A1 XCR>' to prod dfce- the first line- I 
typed DATES <CR> and RAISINS <CJR> ' to produce the, secdncl arid third^ 
The remaining characters are all printed byMLS.^ ' 

NLS asks^'Finished?' sin6e more than one substitutio/i can be madt? at a 
time. If I had typed N for no insteacj of 'Y' for yes, it ' wpuld have 
prompted- for another <New WORD>. and <01d V/onD> pair. 



ATLer I typed 'Y' SLS asks lor further confirmation with OK • ' ^' ^ 
to which I responded v/^th a <CR>. ^ V • 

When the command has Veen exe9uted, NLS tells how m^nj? substitutions 
h^ve been irmde. IT' there had been two instances of RAISIhS in- 
statement 2A1, both would have been chanf^ed to read DATES. • - 

Statement 2A1 is now the current statement. Please ' p^int it . i 

-•^ ' - < . . ;■ 

Student tffptng ia underlined. 
''% ■ . " * ■ ^ • ' ■ ' 

^*SK 111 - The otudcnt confuses tha \ command, uhhh 

pnTT r. /* prtnfca.fr/ic current staboment, iiith the / command 

- NUTTu^AT - ''fiff'^e the aurrcnt position of pHc con'irol 

PnTr rT T »'«''*cr.. /teaU;;t»i^ «ta mistake, /.c /umWca wtt/i 

oi n*;M?7, ^"'i otner one-'Character commanda he can 

ZA OATMEAL Pememlfcr. First he trice f uh^i-ch prints the 

previous Btatemoni (2/U, and rimkcs it bc)Bome 
the current statement. ' 

EDIT C:_J . ffext the student depresses the UNF. FiSED 

o»i iJTn.» W.,.r.o key ohich echoes as %^ giving him the neat • 

EDIT C- • ''^<^*''"i'"i (2AJ), making it eurreni. ^ 

' ■ ' . ■ - / *■ ' ■ 

S'?«''!ri"!:^^''?° '»^"y/t?tements. The tMk is evaluated. 

This is what % v/anted you to print: . 

W1.TH DATES " . .,• 1 



Please redo just the printing , ■ 

part of this task. Your file is unchanged. 

HLS'SCIIOLARia rcallif determined to get the student to review 
now to prvnt tno current stat^ement. 



Fig. 4 Dialogue with >TLS-SCUOI.AI^ (cont) 



continuation of TASK ill 

EDIT C: *Uhat command prints the current statefnent?^^ 

. THE USCT PRIflTS THE CURRENT STATEMENT USING* THK 

\ COMMAND. * ^ , - ; ' ^ 



' Failing to rarncmber the pppropria ic one-^chaTactcr,* command y 
the Gtudent ,giv^a up and asks Xor help^, ' 
Qua^tions can be asked any time ^it is t/ie, student 'a turn to type. 
NLS'-SCllOLAn answers this question by retrieving information from 
' its own internal, descriptive ^data base. ' / 

• ... ■ ■ " v-^,-. . . ^ • ' / •: , 

^rf .' ■ •> 

><bontinuation of TASK^'ll 

EDIT cj\_ - / 

• 2A1 WITH DATES / * / * 
^B|T Ci' Done OK: y - / 

QUESTIONS? » 

^ ■ • ' . _ " . ■. »» 

The^ student choosoG t^ ask no quentiona at this time. 

Try the Substitute command yourself by subhtitatinR the'word " , 
OAT for CORN in the, st^^tcincnt containing "CORN 4'LAKES" 



TASK Ha f:, 

EDIT C: »V/hafc gfcatement contains ncORN" ?* 

V ' ■ : ' ' 

THE STATEMENT CONTAINING "CORN'' 15: 
2C CORN FLAKES ^ ■ 



\ 

■ > 



The student ^wants to know what the corredt statement number is ' 
for him to type. The system obttiif^G , the g,nswev by ^aottiallu _ 
%ooking'into the studeni^s liork space the ansh>er is not stored 
(it cannot be!) in the internal, data bftse. . > 
The sysvem interprets correctly mast ^f the common 
parapkras0e' df 4^his type of quesiioft, e»g* 

^Whatis the statement containing 0 " 

'^What statemenP contains 

^Uhere is .^.'^ etc* ► 



Pig. 4 Dialogue with rW^S-SGHOLAR (Cbht) 



EDI>r C: Substitute C: Word in C: Statement at A: gC; 
<Hew WORD> T: CQMi 

<01d V/ORD> T-: OAT; ^ 
Finished? Y/N: Y 0K:_; 
Substitutions made: 0 
EDIT C: Done OK; ; 



The" student switohca^arouhd the na^ an*d thg - '^d wcrdQ , 
BO no cliange is made. . ^0^^' 



You should have altered the file 
so that this part, of it 

2B CREAM OF WHF.AT 
— ■ — gC fcOnn FLAKKS 1— , 



ULS-SCiiOLAIi shovm^ihc student 



wblap the result this conk 
should have bec^fpitf he had: 



riiha >id 



pnrfnrmnd tfie t^.k onrr^ccll 



would look like thin: 
2B CHliAll OF WHI^AT 
• * 2C OAT FLAKKS 
3 • • • 

But y.ou did riot allcr the file/ 

Please try to re^pair your 
mistake. The file is as you left it, 



as opposed to rcisult, which 

ho obtained. ':^fh. phis 
information/ ^0 student f 
can "dehurj" li^p fpcryovvtancb. 
of the task, M V - 

■w- :■ ' 



continuation of TASK ^2 

EDIT C: *Ht)w does one do this?* 



The stuS^^t losfi and 
asks fc^^hel^. 



THE USER TYPES 'SUBSTITUTE', FOLLOWED BY 'WORD'fr^y 
'STATEMENT' * '^r' ✓pus 'hat' ✓pds * fei"^ 



2C , <CR>, OAT', <CR>, 



'CORN', <CR>, 'Y', <CR>, 'DONE'., AND <CR>, 



NLS' 
to i 



conlginuation of TASK ^2 f 



EDIT C: »Please do it 



R can show him what 
aqi well as. . . 



for me* 



.Motially do it for him! 



EDIT C: (Substitute ) C: (Word ) in C:. (Statement ) at A: (2e)(<CR>): 



<New WORD> T: (OAT)(<CR>); 

<0;d WORD> T; (CORN)(<CR>); 
Finished? Y/N: (Y), OK: (<CR>); 
Substitutions made: 1' 
EDIT C : (Done ) OK : (<CR>) ; 



Fig, 4 Dialogue with NI-S-*fefcHOIAR 
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tight inbegrWtion within a wording erivironment makes . NL5-SCH0LAR a 



powerful tutorial CAI system. 



Tutoring DiagAostic Skills 



j SOPHIE (13) reflects an attempt to extend ranhhllgJ^^ n^^,i^n of 
_.mixed"initiati CAT/ fo^^--^fed're^-"ptIrpose of encouraging a gwicjer rati^ of 
Student initiatives.* Unlike previous tutorial systems which attempt 
to mimic the \roJes of a humar^ teacher, SOPHIE tries\>o 'create a 
"reactive" envirbnment in which the studpnt learns by try'ing out his 
ideas rather than by dinstruction . To this ^nd, SOPHIE incorporates 
a "strong"* mpdel\ of its kn^ledgd domain along with numerous 
heuristic strategies for answering a studisnt 's. que^ions, providing 
.^im* with critiques\ of his cxirrent solutionr? pathSy and generating 
.alternativje theories to his current hypotheses.: In essence, SOPHIE 
enables a student to. have a onerto-one relationship with an "expert? 
who helps the student create, experiment with, , j^d debug his ^own 
ideas. 



.SpPHTE/s expertise is derived from an efficient and powerful 
,inferencing scheme that uses multiple representations of ^knowledge 
including (a)* simulatlybn models of the domain (b) procedural 
specia[lists which coot^in logical skills and heuristic stra^tegies 
for using these mQdels,\ and -(c) semantic nets,, for v encoding 
time-invarjant : factual kn'^dwledge . - The power a/d generality of 
SOPHIE stems, in part , fr^oitti the ^synef gism obtarned by focusing the 
diverse capabilities of \the procedural specialists on the 
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4- . y • ■ • . ^ . ■ ■ 

"intelligent"- manipulation^ execution,^ and interpretation of its 

simulation models. . . ' : 

■■I * * • • ■ ■ 

4 In the basic scenario, SOPHIE acts as an electronics ""lab 

' ' \ " •■•■^ . * ^ ■ 

instructor who :*ielps the stud^pt trangform his classroom knowledge 

• ■ • », . ■ ^ ■ • ■ ■ ■• 

of ei6ctronics into an ^ experiential, intuitive knowledge qf its 

. . . ' . . ^ • ^ • "•■ i 

meaning and application, ^It does this ' ■ interacting with the 

student while he is , debugging a maifunctiodihg piece of equipment 

(19). The studetit can perform any sequence of measurements, ask 

either specific questions ' about the ^ i'mplications of \hese 

measurements for more general hypothetical questions, and even ask 

for advice about what to consider next , ^ikren what* he has discovered*^ 

thus far , ^At any : time JSOPHIE may encourage the student to make a 

guess as to what .hf thinks might 'be wrong given the measurements he 

has made thys' ?ar. If he ^oes, SOPHIE will evaluate his hypothesis 

by taking into consideration all the information h'e should have been 

able to derive from hife current set of leasureinents. If any of this 

information "is logically contradicted by the hypothesis, SOPHIE 

identifies and, explains these contradictions. Likewise SOPHIE can 

Judge the merits of any particu-lkr measurement with respeet to the 

pr.iar sequence of ' measurement^ he has made. For example, hi S; new 

measurement may be ^logically redilndantNln the sens! that . -no- new 

information can possibly be derive^ from lit (ao extremely wwmplex 

task to determine SOPHIE clii? klso decide if this measurement 

... - . ■ ■ I * ;r • . • ' 

perrorm^ a reasonable split of the hypothesis spaoe'^bf possible 

^f^^lts 'which have not yet been ruled out by prior measurements. ^ 
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It shoialtl be noted that the scenario CQnta'ins quite a variety 
of logical/'^sks (i.e,, hypothesis evaluation, hypothesis formation, 
redundancy checking, hypothetical question answering) each ohe of. 
iwhich requires a substantial amount of deep logical inferencing. 
One of the basic challenges in constructing SOPHIE was creating an 
inference system which could perform this wide range. of tasTjS; 
efficiently (so that it could be used in real. time) and at the same 
time have it be robust in the sense pf hand^ling' all realistic 
queries, " 



Because ,i^PHIE was- designed as an environment in which students 
Q0ul6 create and articulate ideas, it was necessary to have a 
powerful>.natural. language processor tcf communicate with students. A 
student will become frustrated if he has to try several ways of 
expressing ^n idea to get a response. In addition, he will becbme^ 
bored if there is a long delay (say 10 secW Before the system 
replies. And because stude^nts begin to assume the stystem ©hares 

their "world-view", SOPHIE must cope With contextually-^dependent 

/ . • - ■ 

references, deletions*, and ellipSses. .SOPHIE's natural language 

• - ^- ' ■ ' . ■ ' ■ ■^ 

processor is based upon a "semantic grammar" technique, in .which 

' • ■ f 
concepts like ."measuremen.t" or. ^circuit elements trigger 

exfSectations about what thikgs should app^earMn the student^s input^p 

SOPHIE .-has dem^strated that natural language processing has 

advanced faij enough to deal with these^three kinds of difficulties 

well enough to bufld fAenJflly, but sophisticated tu|;oMal systems. 
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. THE FUTURE OF INTELLIGENT CAI 

- The thrust of this paper has, been to show what «ind of 
capabilities are now available for building genuinely intelligent 
CAI systems. The domain of such* systems is virtually unlimited; it 
is not restricted, feer example, to . drill and practice or 
mathematics. The language ca^bilities of current systems are not I 
equal to those of a human, nor will they be in the foreseeable \ 
future, but they are good enough to sustain practicaij, . systems. 

The Plato system (6) has .shown that it is possible to biiild 
both ^ interesting and cost-effective QAi systems in a time-shared 
computer environment. They have accomplished this by using a 
variety of teaching technictues: the Socratic methbd, generative CAI, 
games and simulatio,ns, programmed Instruction, etc. intelligent CAI 
Is .an -attempt, to go beyond the teohnology in the Plato system to 
explore how to build greater intelligence * into * tutorial systems, • 
.While . at the satne time utilizing many of the educational techniques 
employed so successfully in P>ato. 



Intelligent CAI systems "are now both costly to build . (aboye 
$100,000) and to use (about $10-$20 per fiourj. But, the cost of 
computing continues to decrease whii« teacher's salaVies are rising. 
Hence the cost of running \si*oii systems should be; d^^^^ 

comparison to the cost, of human tutoring Vithin a * short time, 

", ■ ■ . ■ ...V . . .: \ ■ ■ ,' ■ : ; ■ - 

especially vhere there are few skilled teachers available , as with 

teaching computer text-edit^^g. The effective cost of building iuch 

systems depends on how mi^ch t^iey are used* If they ape uste^ 



... 
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heavily, then the large cost of building' them will be worth the 
investment; otherwise not. It is at least possible that one of the 
current systems ^^11 be^ used . enough to justify the development 
expense, though they were built only as prototype systems. The test 
though will be the development of such" a system for a school setting 
.Where large nqmbers "of people are being taught. 

The payoff in intelligent CAI comes from personalizing the 
learning process. Personalization is effective in many ways: by 
forcing the student to participate in learning; by teatshing at the" 
level '-of his individual knowledge; 'by providing a setting where the 
student can try out his own ideas and make mistakes; by frfeeing the 
student from peer pressure; by addressing the student 's individual 
confusions, etc. These advantage's make it worthwhile to give 
intelligent CAI a serious trial. ... 

- - - . - * ' 

t ■ . ' •• - ■ 
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Understanding, D.G. Bobr^w ^and A. Collins, Eds. (Academic 
Press, New tork, 1975) p, ill. J.S. Brown, R,R. Burton, ' and 
A.G. Bell, Inti:^ <L, Man-MaoH. Stud. , in press (1975).^ 
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A. Collins and E.H. Warnock, BBN Report Nq. '28ia*(Bolt 
' Beranek & Newman, Cainbridge, Mass., 1974). 

.:I5. A. Paivio, Im agery aM Verbal Prooesses . (Holt, Rinehart , and. 
; ' Winston, New york, 1971). 



; il6. D.A. Norman, Memory and Attention . (Wiley, ( NeW York, "1969). 
; BQwer. in Cognition- in Learning ai^ ^Aefe . L-.W. arpgg. 

:- i : CMilpy, New YorkV 1972) p.. 51. 

,^7. M.C. Grignetti, C. Hausmann, and L, Gould = in Proc. gf 
; • National gpmpMtgr .Sanf^- (Anaheim, Calif. , ' 1975) p.. 775. m.c. 
Grignetti, L. Gould, A. Ga Bell, C*L. Hausmann, G. Harris, 
and J.J. Passafiume IM Report JJa*. '2969 . (Bolt Beranek & 
Newman Inc., Cambridge, Mass., 19f4). 



18. Other examples are reported* in A. Barr, M. Beard, and R.C. 
. Atkinson, TR^gjS^ (Psychology and Education Series, Stanford U. 
Stanford, Calif. , 1974) and'ln D.R. Centner, M.R.. Wallen aqd 
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Bolt Berarielc and Newman Inc^. 



P.L.* Miller, A Computer-based . System for Studies in . Learning . 

(Center for Human Information Processing, UCSD, LaJolla , Calif . , 

/ • ■ 

• 1974): . ^ . 

19. Although the domain of knowledge under ^v/nsid«ration is 
electronics, the reasoning and linguistic paradigms underlying 

, SOPHIE are applicable to many domains ^utside of electronics. 

• » . • * ■ . , 
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